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(57) Abstract 



An apparatus and method are disclosed for displaying three dimensional representation or worlds and navigating through such worlds. 
In an apparatus embodiment, a viewer user interface (100) adapted to present information that is organized into a discrete chunk of space 
to a user is disclosed. The information potentially includes links and/or objects that are associated with specific locations in the space. The 
viewer user interface includes a first viewport (103) for presenting a three dimenstonal representation of the space from the viewpoint of a 
position within the space, and a second viewport (149) for presenting. a two dimensional birdseye view representation of the discrete chunk 
of space. The viewer further includes a third viewport (109) for presenting an identifier that the discrete chuck of space is associated with. 
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Systems and Methods for Displaying Three Dimensional 
Representations and an Associated Separate Web 
5 Window 

Background Of The Invention 

The present invention relates generally to mechanisms for displaying three 
dimensional representations or virtual reality worlds. More specifically, the present 
10 invention relates to method and apparatus for viewing such three dimensional worlds. 

One conventional mechanism for creating and displaying three dimensional 
(3D) worlds implements virtual reality modeling language (VRML). Conventionally, 
VRML is used to "model" three dimensional worlds using simple polygon structures. 
The first version of VRML allows for the creation of virtual worlds with limited 
15 interactive behavior and is currently the most popular foundation for creating 
interactive 3D multimedia on the Intemet. As will be appreciated by those skilled in 
the art. International Standard ISO/IEC 14772 describes the standard VRML file 
format. 

Worlds created using VRML can contain objects which have hyper-links to 
20 other worlds, HTML documents or other valid MIME types. When the user selects an 
object with a hyper-link, the appropriate MIME viewer is launched. When the user 

! 
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selects a link to a VRML document from within a correctly configured WWW 
browser, a VRML viewer is launched. 

Although the conventional VRML viewer work well under certain conditions, 
it has associated disadvantages. For example, if the 3D world is relatively large, it is 
difficult to orient oneself within the world. That is, as one navigates within the 3D 
world, it is easy to get lost. This is partially because standard VRML viewers do not 
provide any frame of reference for where you are currently located within the VRML 
3D world. Additionally, even when a 3D world is relatively small in size, 
navigational problems may still arise when certain portions of the 3D world have a 
similar appearance. In this case, it is especially difficuh to distinguish between 
similar portions of the world and navigate to a desired location of the 3D world. 

Navigational problems are compounded since each VRML world may have an 
infinite size. In other words, there are no constraints placed on how large a VRML 
world may be. Thus, VRML world builders may build relatively large and complex 
worlds. As VRML worlds becomes larger and more complex, it becomes 
increasingly more difficult to view and navigate through such gargantuan worlds. 
Finally, when a VRML world is relatively complex, it requires a relatively long 
rendering time since each portion of the 3D world is analyzed and rendered from 
polygon structures. 

In view of the foregoing, there is a need for improved mechanism for viewing 
and navigating through 3D worlds. 
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Summary Of The Invention 

Accordingly, the present invention provides an apparatus and method for 
displaying three dimensional representations, spaces, or worlds and navigating 
through such worlds. In general terms, the navigation and display mechanisms and 
techniques provide one or more frame of references that indicate where the user is 
currently located within the three dimensional world. 

In one embodiment, a viewer user interface adapted to present information that is 
organized into a discrete chunk of space to a user is disclosed. The information 
potentially includes links and/or objects that are associated with specific locations in the 
space. The viewer user interface includes a first viewport for presenting a three 
dimensional representation of the space from the viewpoint of a position within the 
space, and a second viewport for presenting a two dimensional birdseye view 
representation of the discrete chunk of space. The viewer further includes a third 
viewport for presenting an identifier that the discrete chuck of space is associated with. 

In an alternative embodiment, a viewer user interface adapted to present 
information that is organized into a discrete chunk of space having a plurality of sections 
to a user is disclosed. The information potentially includes links and objects including at 
least some of images, sound clips, video clips and sprites that are associated with specific 
locations in the space. The viewer user interface includes a first viewport for presenting 
a three dimensional representation of the space from the viewpoint of a moveable 
reference position within the space, and a second viewport for presenting a two 
dimensional birdseye view representation of the discrete chunk of space. The second 
viewport is arranged to indicate the current reference position. The viewer also includes 
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a third viewport for presenting a identifier that the discrete chuck of space is associated 
with, and an entry bar arranged to identify the discrete chunk of space that the first and 
second viewports are presenting. The viewer further includes a local Hnk viewer 
arranged to identify links in selected locations that are adjacent to the current reference 
5 position, and a status box arranged to present status information. 

In anodier aspect of the invention, a viewer user interface adapted to present 
information that is organized into a discrete chunk of space having an array of tiles to a 
user is disclosed. The information potentially includes links and assets including at least 
some of sound clips, video clips and sprites that are associated with specific locations in 

10 the space. The viewer user interface includes a first viewport for presenting a three 
dimensional representation of the space from the viewpoint of a moveable reference 
position within the space, and a second viewport for presenting a two dimensional 
birdseye view representation of the discrete chunk of space. The second viewport is 
arranged to indicate the current reference position. The viewer also includes a local link 

15 viewer arranged to identify links in selected tiles that are adjacent to a current reference 
tile that includes the current reference position. 

In a method aspect, a method of displaying a tliree dimensional representation is 
disclosed. A first three dimensional view of the three dimensional representation is 
changed to a second three dimensional view of the three dimensional representation after 
20 receiving an input signal indicating movement within the three dimensional 
representation. A first position of a reference object within a discretely sized two 
dimensional representation is changed to a second position. The reference object is 
associated with the three dimensional representation. The first position corresponds to 
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the first three dimensional view, and the second position corresponds to the second three 
dimensional view. 

In another aspect a display system for displaying a three dimensional world is 
disclosed. The display system includes a three dimensional display window arranged to 
5 output at least a portion of the tliree dimensional world, and a two dimensional display 
window arranged to output a static two dimensional representation of the three 
dimensional world that includes a position indicator associated with the portion of the 
three dimensional world that is displayed in the three dimensional display window. 

In yet another embodiment, a computer readable medium containing program 
10 instructions for displaying a three dimensional representation is disclosed. The computer 
readable medium includes computer readable code for changing a first three dimensional 
view of the three dimensional representation to a second three dimensional view of the 
three dimensional representation after receiving an input signal indicating movement 
within the three dimensional representation and computer readable code for changing a 
15 first position of a reference object within a discreteK sized two dimensional 
representation to a second position. The reference object is associated with the three 
dimensional representation, the first position corresponding to the first three dimensional 
view and the second position corresponding to the second three dimensional view. The 
computer readable medium also include computer readable medium for storing the 
20 computer codes. 

These and other features and advantages of the present invention will be presented in 
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more detail in the following specification of the invention and the accompanying 
figures which illustrate by way of example the principles of the invention. 



Brief De scription Of The Drawtnc-^ 



The present invention will be readily understood by the following detailed 
description in conjunction with the accompanying drawings, wherein like reference 
numerals designate like structural elements, and in which: 

Figure 1 is diagrammatic representation of a viewer system for displaying 
three dimensional representations in accordance with one embodiment of the present 
invention. 

Figure 2 is a screen shot illustrating the viewer of Figure 1 as it displays a 
particular three dimensional representation in accordance with one embodiment of the 
present invention. 

Figure 3 is a screen shot of a pull down menu that is selectable from the entry 
box of the viewer in accordance with one embodiment of the present invention. 

Figure 4 is a screen shot illustrating an option pull down menu of the viewer 
in accordance with one embodiment of the present invention. 

Figure 5 is a screen shot illustrating a world list display box within the viewer 
in accordance with one embodiment of the present invention. 



6 
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Figure 6 is a screen shot illustrating a settings input box that is displayed when 
the settings button, for example, of the viewer is selected in accordance with one 
embodiment of the present invention. 

Figure 7 is a flowchart illustrating a rendering procedure implemented by the 
5 viewer of Figure 1 in accordance with one embodiment of the present invention. 

Figure 8 is a flowchart illustrating the operation of Figure 7 for rendering of 
the three dimensional viewport of Figure 1 in accordance with one embodiment of the 
present invention. 

Figure 9 is a flowchart illustrating the operation of Figure 8 for rendering 
10 MIDI blended/unblended sound objects in accordance with one embodiment of the 
present invention. 

Fi gure 10 is flowchart illustrating the operation of Figure 8 for rendering wave 
blended/unblended sound objects in accordance with one embodiment of the present 
invention. 

15 Figure 1 1 is a flowchart illustrating the operation of Figure 8 for rendering a 

three dimensional display in accordance with one embodiment of the present 
invention. 

Figure 12 is a flowchart illustrating the operation of Figure 1 1 for rendering 
the background into one or more buffers in accordance with one embodiment of the 
20 present invention. 

7 
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Figure 13 is a flowchart illustrating the operations of Figure 1 1 for rendering 
the ceiling and floors in accordance with one embodiment of the present invention. 

Figure 14 is a flowchart illustrating the operation of Figure 11 for rendering 
walls and sprites on walls within the three dimensional representation in accordance 
with one embodiment of the present invention. 

Figure 15 is a flowchart illustrating operation of Figure 1 1 for rendering sprite 
objects that are not in locations that contain walls in accordance with one embodiment 
of the present invention. 

Figure 16 is a flowchart illustrating the operation of Figure 7 for updating the 
local links viewer in accordance with one embodiment of the present invention. 

Figure 17 is a flowchart illustrating the operation of Figure 7 for updating the 
two dimensional viewport in accordance with one embodiment of the present 
invention. 

Figure 18 is a flowchart illustrating a process for updating the world list in 
accordance with one embodiment of the present invention. 

Figure 19 is a flowchart illustrating the process for loading a selected link in 
accordance with one embodiment in the present invention. 

Figure 20 is a flowchart illustrating the operation of Figure 19 for loading the 
three dimensional representation in accordance with one embodiment of the present 
invention. 
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Figure 21 is a flowchart illustrating the process for sending and receiving 3D 
email in accordance with one embodiment of the present invention. 

Examples of 3D worlds and corresponding web displays are illustrated in 
Figures 22-27 in accordance with one embodiment of the present invention. 

Figure 28 is a flowchart illustrating a process for creating a token file in 
accordance with one embodiment of the present invention. 

Figure 29 illustrates how the token is then installed within the associated 
builder in accordance with one embodiment of the present invention. 

Figure 30 is a flowchart illustrating the process for building a 3D world with 
the token in accordance with one embodiment of the present invention. 

Figure 31 is a flowchart illustrating a process of rendering the token within the 
viewer in accordance with one embodiment of the present invention. 

Detailed Description Of Specific Embodiments 

Reference will now be made in detail to a specific embodiment of the 
invention. An example of this embodiment is illustrated in the accompanying 
drawings. While the invention will be described in conjunction with this specific 
embodiment, it will be understood that it is not intended to limit the invention to one 
embodiment. On the contrary, it is intended to cover alternatives, modifications, and 
equivalents as may be included within the spirit and scope of the invention as defined 
by the appended claims. In the following description, numerous specific details are 
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set forth in order to provide a thorough understanding of the present invention. The 
present invention may be practiced without some or all of these specific details. In 
other instances, well known process operations have not been described in detail in 
order not to unnecessarily obscure the present invention. 

5 In general terms, the present invention provides mechanisms and techniques 

for displaying three dimensional (3D) representations, spaces, or worlds and for 
navigating through such 3D representations. A graphical user interface is provided 
within a "viewer" for displaying and navigating through the 3D representations. 
Among other features, the viewer provides mechanisms for determining a current 
ID location within the currently displayed 3D representation. Tlie frame(s) of reference 
of the viewer allow the user to navigate freely through the 3D representation without 
getting lost. 

In one embodiment, each 3D representations is in the form of a discretely 
sized chunk and may be linked to other 3D representations. Thus, the viewer 

15 facilitates navigation through one chunk at a time. In other words, the viewer loads 
and displays discretely sized 3D representations. In the illustrated embodiment, the 
3D world corresponds to a 16 by 16 array of tiles, locations, or sections. Each tile is 
associated with one or more objects and/or links that are executable or displayable 
within the 3D world. A column of viewable space within the 3D world may 

20 correspond to a particular tile and its associated objects and/or links. 

By way of example, a particular tile of a populated construction site may be 

associated with image objects that are displayed within the associated column of 

space of the 3D world. The particular tile may have a floor image, such as a brick 

10 
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sidewalk portion, that is displayed within the corresponding location within the 3D 
world. The particular tile may also have an associated 3D object, such as a grocery 
store, that is displayed on top of the floor image in the 3D world. In other words, a 
particular viewable space within the 3D world may be associated with a tile having 
one or more objects that are displayed at various viewing levels within the associated 
viewing space. Besides having an associated floor image and a 3D object, the 
particular space or tile may also include an overhead image, such as a ''cloudy sky" 
image that is displayed over the floor and 3D object images. 

Besides including images, a particular space may also include various objects 
that are executed by various software applications. In the above described example, 
the particular space and tile may include a URL link to a grocery company's web site. 
By way of other examples, the space may include any type of object, such as a video 
clip or a Microsoft Word document. 

The viewer may also allow access to other 3D representations through various 
mechanisms. For example, the viewer may provide mechanisms for jumping from a 
first 3D representation to a second 3D representation that is linked to the first 3D 
representation. Thus, even though the viewer may only display a single 3D chunk at 
one time, there is no limit to how many 3D representations may be accessed and 
displayed. 

Figure 1 is diagrammatic representation of a viewer system 100 for displaying 

three-dimensional representations in accordance with one embodiment of the present 

invention. As shown, the viewer 100 includes a three dimensional viewport 103 and 

a two dimensional viewport 106. The three dimensional (3D) viewport is configured 

1 1 
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to display a portion of the three-dimensional representation as the user navigates 
through the 3D representation. The 3D viewport may be extendable such that the 
entire viewer 100 space is filled by the 3D viewport. 

As the user moves through the 3D representation, the user's immediate 
surroundings are displayed within the 3D viewport 103. The surroundings are 
displayed as a virtual reality type environment, which is based on the user's position 
within the 3D representation. The 3D environment may include various displayed 3D 
objects, and one or more of the 3D objects may be linked to one or more executable 
objects {e.g., a web page or another 3D representation) 

The two dimensional (2D) viev^ort 106 displays a 2D perspective of most or 
preferably all of the 3D representation. For example, the 2D viewport 106 displays a 
birdseye view of the entire 3D representation. That is, the 2D viewport 106 provides 
the user with a 2D map of the 3D representation. The 2D viewport 106 may also 
include a current location indicator 149 for providing the user with a frame of 
reference for the user's current location within the 2D representation, as well as the 
3D representation. The current location indicator 149 within the 2D viewport 106 is 
tied to the user's position within the 3D representation. In other words, as the user's 
view and environment changes within the 3D viewport 103, the current location 
indicator's position 149 within the 2D viewport will correspondingly change to 
indicate the user's new position within the 3D representation. 

Additionally, the current location indicator 149 may indicate a viewing 
direction. As shown, the current location indicator 149 is in the form of an arrow that 
points towards the western portion of the 2D viewport 106. In this example, the 
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current location indicator 149 indicates that the user is looking towards the western 
portion of the 3D representation. 

In sum. the current location indicator 149 provides the user with a useful 
mechanism for determining where they are currently located and in which direction 
the user is looking within the 3D representation. Thus, the user does not have to 
wander aimlessly within a 3D representation; they can simply determine their position 
and viewing direction within the 3D representation by checking the current location 
indicator's position and direction 149 within the 2D representation. 

The viewer 100 may also include other features for allowing tlie user to 
quickly assess their current position within the 3D representation. For example, the 
viewer 100 may also include a local link viewer 113. The local link viewer 113 is 
configured to display links that are relatively near or adjacent to the user's current 
position within the 3D representation. A link may reference any type of executable 
object. For example, a link may reference a web page, a video clip, or another 3D 
representation. 

Links that are positioned near the user may be displayed in the local link 

viewer 113. In the embodiment shown, links are displayed along two aisles down 

which the user is navigating and each aisle displays up to four links. This 

embodiment is particular useful in shopping applications. As the user walks down 

virtual shopping aisles, nearby items on each aisle are displayed as links. 

Altematively, links may be displayed that are within a certain distance from the user's 

current location. By way of another example, the local link viewer 1 13 may display 

only the links that are positioned closest to the user's current location. 

13 
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In one embodiment, each column of space within the 3D world is associated 
with a populated construction site tile. Each column of space and associated tile may 
have, one or more object or object links. Thus, when a user is positioned within a 
specific column of space within the 3D world, she is near or adjacent to other 
5 columns of space and associated tiles. Thus, the user may be positioned near or 
adjacent to other attribute instances that were placed on the other near or adjacent 
tiles. If these nearby attribute instances are in the form of links, they may be 
displayed within the local link viewer 113. 

A status box 1 1 5 may also be provided for displaying useful information to 
10 the user. For example, when a pointer, such as a mouse, is moved over a particular 
location within the 3D viewport 103, the status box 115 may identify any links or 
objects that are associated with the location. For example, a name of the associated 
link or object may be displayed within a text box positioned over the associated 
location within the 3D representation. 

15 Additionally, the viewer 100 may include an entry box 111 for directly 

selecting a link within the 3D representation. When a link that references another 3D 
representation is selected, the 3D viewport 103 switches from displaying the current 
3D representation to displaying a second 3D representation that corresponds to the 
selected link. Alternatively, when a link that references an executable object 

20 associated with a particular software application is selected, the link is loaded by the 
associated application. 

One or more 3D representations may be grouped together and associated with 
an emblem. As shown, an emblem 109 is displayed by the viewer 100. The emblem 

14 
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109 indicates to which group or category the currently displayed 3D representation 
belongs. A 3D representation may also belong to a hierarchical list of 3D 
representations. Within the hierarchical list, a particular 3D representation may be 
associated with a parent 3D representation. 

The viewer 100 may also provide a mechanism for displaying one or more 
hierarchical lists. As shown, the viewer 100 includes a world button 127. When the 
world button 127 is selected, a hierarchical list is displayed that includes several 3D 
representations that form part of a tree structure. A user may select and go to a 
particular 3D representation by selecting a 3D representation from the hierarchical 
list. The selected 3D representation is then displayed within the 3D viewport 103. 

A particular 3D representation may be defined as a home 3D representation. 
By way of example, a user may define a 3D representation of their hometown as their 
"home" 3D representation. Accordingly, a home button 129 is provided within the 
viewer 100 so that the user may quickly jump back to their defined home 3D 
representation. When the user selects the home 3D representation via the home 
button 129, the home 3D representation is displayed within the 3D viewport 103. 

The currently displayed 3D representation may include several predefined tour 
routes. When a particular tour route is selected, the user's travel through the 3D 
representation is automated into a predefined route having predefined events that are 
defined by the selected tour route. A tour route may be selected in any suitable 
manner. For example, a tour may be automatically activated when the user navigates 
over a particular location within the 3D representation. As shown, the viewer 100 
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also includes a tour button 131 for selecting a tour associated with the currently 
displayed 3D representation. 

As described above, particular locations within the 3D world may be linked to 
other worlds or executable objects, such as URL sites or video clips. As a user travels 
between these different links and objects, mechanisms may be provided for jumping 
back to previously traveled links or forward again to previously traveled links. For 
example, a user may travel from a first world to a second world to a third world. The 
user may then wish to travel back to the second world. Thus, a previous button 
(shown in Fig. 2 as 137) may be selected, and the user jumps back to the second 
world. The user may then wish to travel forward again (e.g. to the third world), and 
this may be accomplished by utilizing a next button (shown in Fig. 2 as 139). When a 
link is loaded by a web browser and the user travels from the first link to other links 
via the web browser, the user may wish to jump all the way back to the 3D world that 
was displayed before the first link was loaded and displayed. Thus, an exit button 
126 is provided to allow the user to travel all the way back to the 3D world from 
which the link was loaded. 

The viewer 100 may also include various mechanisms for choosing how the 
3D representation is displayed and how associated navigation tools within the viewer 
100 function. As shown, the viewer 100 includes a settings button 135 and a speed 
button 133. The speed button 133 allows the user to quickly toggle through various 
speeds for navigating within the 3D representation. As the user selects different 
speeds, it appears as though the user is traveling at different rates within the 3D 
representation, as displayed within the 3D viewport 103. 

16 
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Other characteristics for displaying the 3D representation may also be selected 
or modified through the settings button 135. This feature, along with other features, 
of the viewer 1 00 are further described below. 

Figure 2 is a screen shot illustrating the viewer 100 of Figure 1 as it displays a 
5 particular 3D representation in accordance with one embodiment of the present 
invention. As shown, the 3D representation is in the form of a mail. A corresponding 
birdseye view of the entire 3D representation is displayed within the 2D viewport 
106. 

The currently displayed three dimensional representation is associated with a 
10 specific emblem 109. As shown, the emblem 109 is in a form of a mall exterior. 
Other 3D representations may be associated with the same emblem 109. For 
example, 3D representations that are associated with the mall emblem 109 may 
include a mall court yard, a number of shops within the mail, a mall information 
booth, and the currently displayed mall interior. 

15 Other display areas within the viewer 100 contain information associated with 

the currently displayed 3D representation 103. As showoi, the entry box 1 1 1 contains 
a reference to the name of the currently displayed 3D representation (e.g., lobby). 
The local link viewer 113 displays links that are in proximity to the current location 
within the 3D representation. Any suitable number of links may be displayed that are 

20 near or adjacent to the current location. Preferable, the displayed links' positions 
within the local link viewer 113 correspond to the links' position within the 3D 
representation. As shown. Nabisco store URL links are displayed along the right side 

17 
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of the local link viewer 113. Correspondingly, a Nabisco store 147 is displayed 
within the 3D representation 103 to the right of the 3D viewport 103. 

As shown, a text box for the Nabisco store link is also displayed on the 
Nabisco store front. In one embodiment, a link text box is displayed when an input 
5 device, such as a mouse controlled pointer, is poshioned over a link in the 3D 
representation. The link name may also be displayed within the status box 115, as 
shown, when the input device is positioned over or near the link in the 3D 
representation. 

In one embodiment, the 3D viewport 103 includes a view direction marker 
10 145. This marker 145 indicates a viewing direction within the 3D representation 103. 
As shown, the marker 145 indicates that a northern portion of the 3D representation 
103 is currently being displayed. This marker 145 may be useful when the user is 
traveling in the direction that is opposite to the viewing direction. For example, if the 
user is walking in a northern direction within the 3D representation and looking 
15 toward the western portion of the 3D representation, the marker 145 is positioned 
along the left side of the 3D viewport, 103, 

Figure 3 is a screen shot of a pull down menu 302 that is selectable from the 

entry box 111 of the viewer 100. As shown, the pull down menu 302 includes a list 

of links associated with the currently displayed 3D representation. Additionally, the 

20 pull down menu 302 includes a link to the currently 3D representation, which link is 

highlighted. The links displayed within the pull down menu 302 may be selectable. 

When a link is selected from the pull down menu 302, a 3D representation that 

corresponds to the selected link is displayed within the 3D viewport 103. Thus, the 

18 

BNSDOCIO: <WO_0020987A2J_> 



wo 00/20987 PCT/IB99/01 799 

displayed 3D representation may be changed to another 3D representation by 
selecting a link to another 3D representation from the pull down menu 302. 

Links may also be selected via an option pull down menu 402 that is 
accessible by a right click of the mouse button within the 3D viewport 103. Figure 4 
is a screen shot illustrating an option pull down menu. 402 of the viewer 100 in 
accordance with one embodiment of the present invention. As shown, links may be 
selected directly from the options menu 402. For example, the adjacent Nabisco link 
may be selected from the pull down menu 402. Additionally, the links associated 
with the currently displayed 3D representation may be accessed and selected through 
either the link pull down menu 302 or the option pull down menu 402. 

Thus, the viewer 100 may provide several alternatives for selecting a link 
associated with the currently displayed 3D representation. A link may be selected 
through the two pull down menus (302 and 402). Alternatively, a link may be 
selected through the "world" button 127. Additionally, a link may be selected by 
moving over a posidon within the 3D representation that includes a link or by 
clicking on such position. By way of a final example, a link may be selected by 
clicking on the link within the 2D representation displayed in the 2D viewport 106. 

The 2D viewport 106 may include any suitable mechanism for allowing 

access and selection of a link associated with the currently displayed 3D 

representation. In one embodiment, a link may be selected by clicking once or 

clicking twice on a position within the 2D representation that includes a link. A 

single click immediately activates the link. For example, if the link references second 

3D representation, the second 3D representation is displayed within the 3D viewport 
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103. When the selected link references an executable object, the object is loaded and 
executed by the appropriate application. In contrast, when a user double clicks on a 
link in the 2D representation, the user first walks through the currently displayed 3D 
representation to the selected link, and then the object referenced by the link is 
activated (e.g., a second 3D representation is loaded and displayed). 

Returning to Figure 4. the options pull down menu 402 may include one or 
more features that are duplicated on the buttons of the viewer (e,g., the world 127, the 
homel29, the tour 131. the "prev" 137, the "next" 139, the speed 133, and the settings 
135 buttons). For example, a previous or next site may be selected; the home 3D 
representation may be selected; a world list may be displayed; the speed may be 
changed; and the settings may be changed through the option pull down menu 402, as 
well as the buttons. 

The option menu 402 allows access to features even when the corresponding 
button is not displayed. In certain modes, the 3D viewport 103 is expanded and 
covers the button area. In this extent, the features that were once selectable through 
the lost buttons are not inaccessible since the same features are available through the 
options menu 402. 

Figure 5 is a screen shot illustrating a world list display box 502 within the 

viewer 100. The world list display box 502 may be activated by the world button 127 

or within the option pull down menu 402. As shown, a hierarchical list of links and 

markers are displayed within the world list box 502. The world list hierarchically 

organizes the different 3D representations. That is, each 3D representation has a 

parent 3D representation or a parent marker. A marker is a label for organizing a 
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group of 3D representations. As shown, a number of 3D representations (e.g., "travel 
in time'\ "grass", "moma", and "music") are associated with the marker "other 
samples." 

The world list display box 502 may include other features for characterizing 
5 the different 3D representations within the displayed hierarchical list. As shown, 
when a particular 3D representation, such as "grass", is selected, a text description 
504 appears below the world list ("meadow with pastoral sounds"). 

Figure 6 is a screen shot illustrating a seUings input box 602 that is displayed 
when the settings button 135, for example, of the viewer 100 is selected. As shown. 
10 characteristics of the 3D representation may be altered within the settings dialog box 
602. A height value of the 3D representation may be selected with a slider 604. The 
height value corresponds to how far the user's view appears to be above the floor 
within the 3D representation. 

A horizon value may also be selected via a horizon slider 606. The horizon 
15 value corresponds to a viewing angle within the 3D representation with respect to the 
ground. For example, if a relatively high value is chosen for the horizon value, it 
appears as if the user is looking toward the ceiling of the 3D representation. In 
contrast, if a relatively low value is chosen for the horizon value, it appears as if the 
user is looking toward the floor. 

20 A distance value may also be selected via a distance slider 608. The distance 

slider 608 corresponds to a viewing angle for the 3D representation. That is. the 
distance value corresponds to how large a portion of the 3D representation is 
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displayed within the 3D viewport 103. A 3D viewport size value may be chosen by 
selecting a corresponding viewport size button 610. The viewport size value indicates 
how. large the actual 3D viewport 103 appears relative to the viewer 100. For 
example, a size value of "full size" results in a 3D viewport 103 that covers the button 
5 display area. 

Objects and/or links are rendered within the viewer 100 using any suitable 
mechanisms. Several embodiments of rendering mechanisms are described below in 
reference to Figure 7 through 17. Figure 7 is a flowchart illustrating a rendering 
procedure 5500 implemented by the viewer 100 of Figure 1 in accordance with one 
10 embodiment of the present invention. 

Initially, temporal images are updated in operation 5510. In other words, any 
images that require an update after a predefined period of time are updated 
periodically when the rendering procedure 5500 cycles back to operation 5610. By 
way of example, objects that are animated require that a new or next frame is 
15 displayed after each predefined time period. 

After the temporal images are updated, a current viewer position is checked to 
determined whether a position change has occurred in operation 5520. If no change 
has occurred, the temporal images are periodically updated in operation 5510 and the 
current viewer position is periodically checked in operation 5520 to determine 
20 whether a position change has taken place. 

However, if a position change has occurred, the current position is determined 
in operation 5530. The 3D viewport is then rendered in operation 5540 based at least 

22 

BNSDCXJID: <WO_00e0987A2_L> 



wo 00/20987 PCT/IB99/01 799 

in part on the determined current position. The local links display may also be 
updated in operation 5550 based at least, in part, on the determined current position. 
Additionally, the 2D viewport 106 may be updated in operation 5560 based at least in 
part on the determined current position. Other features of the user interface of the 
5 viewer 100 may also require updating in operation 5570. For example, some of the 
buttons may be deactivated, such as the "prev" button 137 and/or the "next" button 
139 (as shown in Figure 2, the next button 139 is deactivated). 

Rendering of the 3D viewport is illustrated in Figure 8 in accordance with one 
embodiment of the present invention. Initially, a 3D display is rendered in the 3D 

10 viewport 103 in operation 5542. In other words, any images that are visual are 
displayed within the 3D viewport 103. Next, any sound objects may be rendered, A 
single sound object may be rendered for the current user position, or a plurality of 
sound objects may be blended together and rendered for the current position. The 
blended sounds may be associated with the current position or with positions that are 

15 near the current position. In one embodiment. MIDI sound objects are rendered in 
operation 5544, and wave sound objects are rendered in operation 5546. Although 
the present invention is described in terms of sound objects of MIDI or wave format, 
of course, any suitable sound format may be implemented. Any other type of media 
objects are then rendered in operation 5548, such as video clips or stream video, and 

20 the process 5540 of rendering the 3D representation ends. 

The 3D representation and associated objects and/links are rendered, in part, 
from a world data structure. Objects and links are grouped into attribute layers within 
the data structure. Each attribute layer is partitioned into cells, sections, or locations 
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that correspond to locations within the rendered 3D representation. Additionally, the 
attribute layers may be conceptually layered on top of one another to associate a 
plurality of objects and/or links with a particular location within the 3D 
representation. In other words, a particular location within the 3D representation 
corresponds to a location or "cell" within several attribute layers, wherein each cell 
may contain an object or link. 

In one embodiment, each cell is associated with a particular tile of the above 
described construction site. The different attribute instances that are placed on the 
particular tile are represented within a cell of an attribute layer that is associated with 
the particular type of attribute instance. 

By layering the objects and links together on a particular location of the 3D 
representation, a rich, realistic 3D environment may be provided. Additionally, when 
executable objects and/or links are associated with 3D objects, the resulting 3D 
representation provides a visually appealing and logical mechanism for displaying 
and organizing links and objects. For example, a web page for a music catalog site 
may be associated with a meaningful 3D object, such as a 3D music store, or in a 
larger store, possibly a CD section of the music store. 

Figure 9 is a flowchart illustrating the operation 5544 of Figure 8 for 
rendering MIDI sound objects in accordance with one embodiment of the present 
invention. Initially, for the current position it is determined whether there is a MIDI 
object in a cell of the MIDI attribute layer associated with the current location in 
operation 902. If there is not an associated MIDI object for the current position, any 
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MIDI object that is currently playing is halted or stopped in operation 908 and the 
process 5544 ends. 

However, if a MIDI object is present within the MIDI layer for the current 
position, it is then determined whether the same MIDI object is currently playing in 
operation 904. If the same MIDI object is currently playing, playing of the MIDI 
object is continued in operation 910. In contrast, if the same MIDI object is currently 
not playing, the old MIDI object is stopped and the new MIDI object from the MIDI 
layer is then played in operation 906. Process 5544 then ends. 

Although, only a single MIDI object is described as being played for a 
particular position within the 3D representation, of course, a plurality of MIDI objects 
may be played or a sound object of another sound format may be played for the 
particular position. For example, several sound objects from nearby locations may be 
blended together and played for the current position. 

Blending of nearby sound objects is described as being implemented for wave 
sound objects. Figure 10 is flowchart illustrating the operation 5546 of Figure 8 for 
rendering wave sound objects in accordance with one embodiment of the present 
invention. Initially, locations that surround (or are proximate to) the current position 
are checked for wave objects in operation 1002. For each wave object, a volume level 
and one or more sound direction(s) are determined in operation 5004. For example, a 
left and/or a right channel may be selected for the sound direction. Sound objects are 
then blended using any suitable sound blending technique. For example, alpha 
blended audio may be implemented. 
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Figure 1 1 is a flowchart illustrating the operation 5542 of Figure 8 for 
rendering a 3D display in accordance with one embodiment of th8lp)resent invention. 
In general terms, this process includes rendering of various objects into one or more 
buffers that are associated with one or more portions of the display area {e.g., the 3D 
viev^TDort 103). As shown, operations 1102 through 1108 include rendering the 
background, the ceilings, the floors, and the walls into one or more buffers. The 
rendering of the background is described in more detail below with respect to Fig. 12. 
The rendering of the ceilings and floors are described in more detail below with 
reference to Fig. 13, and the rendering of the walls is described in more detail with 
respect to Fig. 14. 

After the background, ceilings, floors and walls have been rendered, sprites 
that are associated with walls are then rendered into buffer(s) in operation 1 1 10a, and 
sprites that are not associated with walls are then rendered into buffer(s) in operation 
1 1 10b, As described below in reference to Figs. 14 and 15. the two different types of 
sprites are rendered differently. 

After the buffers are filled, the buffers are then output onto the display area 
{e.g., the 3D viewport) in operation 1112. For example, particular locations within 
each buffer correspond to a particular location on the display area. The data within 
the particular buffer location is used to generate a displayed object or image portion at 
the corresponding screen location. The process 5542 then ends. 

Figure 12 is a flowchart illustrating the operation 1102 of Figure 11 for 
rendering the background into one or more buffers in accordance with one 
embodiment of the present invennon. Initially, the left edge of the image on the 



wo 00/20987 PCT/IB99/01799 

display area is determined based on the extent of the 3D viewport and viewer 
direction in operation 1202. 

The extent of the 3D viewport 103 is defined as the viewing area. In one 
embodiment, the extent is the area that is displayed between the left and right of the 
5 computer screen. The viewer direction is defined with respect to the 3D 
representation. For example, if the user or "viewer" is looking towards the northern 
portion of the 3D representation, the direction is defined as north and a northern 
portion of the 3D world is displayed. 

After the left edge of the image on the display area is determined, a current 
10 row of the image from the 3D viewport is then obtained in operation 1204. A left 
edge of the image row corresponding to the 3D viewport row is then determined in 
operation 1206. In other words, a row within the background image is matched to the 
row of the 3D viewport that was obtained in operation 1204. 

After the left edge of the corresponding image row is obtained, pixels are 
15 copied to the 3D viewport row until the 3D viewport row is filled in operation 1208. 
In one embodiment, pixels are sequentially copied from the image into the viewport 
row. For example, if a first viewport row is twice a long as a first image row, the first 
image row and then a second image row are copied into the first viewport row to fill 
up the first viewport row. This technique allows the first viewport row to be 
20 efficiently filled without duplicating the first image row. as compared with 
conventional techniques. 

After the 3D viewport row is full, it is determined whether there are more 

27 

BNSDOCIO: <WO 0Qg09a7A2 I > 



wo 00/20987 PCT/IB99/D1 799 

rows in the 3D viewport in operation 1210. If there are more viewport rows, a current 
row pointer is advanced in operation 1212 such that a next row from the 3D viewport 
may be obtained in operation 1204. Operations 1204 through 1208 are repeated for 
all of the rows in the 3D viewport such that pixels may be copied from the image to 
5 the corresponding 3D viewport rows. When there are no more rows in the 3D 
viewport, the process 1 1 02 ends. 

Figure 13 is a flowchart illustrating the operations 1 104 and 1 106 of Figure 1 1 
for rendering the ceiling and floors, respectively, in accordance with one embodiment 
of the present invention. Initially, a position of a plane within the 3D viewport 103 is 
10 determined relative to the floor within the 3D viewport in operation 1302. Any 
suitable mechanism may be implemented for calculating a position of a plane upon 
which the floor image will be rendered. For example, a perspective transformation 
may be implemented for the floor plane in operation 1302. 

After the plane position is determined, pixels within the section images are 
15 mapped onto the determined plane to form the floor image within the 3D 
representation in operation 1304. Any suitable mapping technique may be 
implemented. For example, sections may be scan-converted into the 3D viewport 
using the section images of color texture. After the sections are scan-converted, the 
ceiling or floor rendering process is complete. 

20 Figure 1 4 is a flowchart illustrating the operation 1 1 08 and 1 1 1 Oa of Figure 1 1 

for rendering walls and sprites on walls within the 3D representation in accordance 
with one embodiment of the present invention. The following operations 1406 

through 1422 are implemented for each column in the 3D viewport 103 (1402) and 
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for each location (e.g,. for each tile) on the line of sight within each column of the 3D 
viewport (1404). Additionally, operations 1408 tlirough 1422 are implemented on 
each location that has a wall (1406). 

Initially, a height of the current wall on the current location is determined in 
5 operation 1408. Next, a column in the wall image that corresponds to the column in 
the 3D viewport is determined in operation 1410. A visible portion of the column in 
the 3D viewport in then determined in operation 1412. This step is done to determine 
whether any other objects within the 3D viewport obscure a portion or all of the wall 
within the 3D viewport. 

10 Visible image column data is then mapped to the corresponding 3D viewport 

column in operation 1414. A distance to the wall is determined and stored for later 
use by the rendering procedure for sprite objects that are not located on a wall in 
operation 1416. It is then determined whether a sprite is located at the current 
location in operation 1418. In other words, it is determined whether a sprite object 

15 and a wail object are located on the same location. If a sprite object exists at the 
current location, a column within the sprite image is then calculated that corresponds 
to the 3D viewport column in operation 1420. The sprite image data is then mapped 
onto the 3D viewport column in operation 1422. 

However, if a sprite object is not present at the current location, the rendering 
20 process of Figure 14 is repeated again for a next column in the 3D viewport starting at 
operation 1402. Likewise, after sprite image data is mapped onto the 3D viewport 
column in operation 1422, the entire rendering process is repeated for the next column 
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in the 3D viewport in operation 1402. After the sprite objects associated with wail 
objects are rendered, the process continues at operation 1 11 Ob of Figure 1 1 . 

Figure 15 is a flowchart illustrating operation 1110b of Figure 11 for 
rendering sprite objects that are not in locations that contain walls. Operations 1 504 
through 1506 are repeated for each sprite object (1502). Initially, screen dimensions 
are computed based on a user's current position and a sprite object's attributes in 
operation 1504. In other words, a bounding box is defined for the sprite object within 
the 3D representation. The spite object attributes may include any suitable attribute 
values for characterizing how the sprite image will be displayed within the 3D 
representation. For example, the sprite attribute values may include an image size for 
the sprite object or a proportion value that indicates whether the sprite object will 
appear stretched or shrunken within the 3D representation. 

After the screen dimensions are computed, the sprite object is clipped as 
necessary to the 3D viewport around any obscuring walls based on information stored 
while rendering the walls (see operation 1416 of Figure 14) in operation 1505. In 
other words, pixels within the sprite image that are obscured by other walls are 
clipped such that they will not be displayed within the 3D representation. 

Whether or not the sprite object is obscured by other walls may be determined 
in any suitable manner. For example, distances between a wall that may potentially 
obscure the sprite object and the user's current location may be compared to a distance 
between the sprite object and the user's current location. Thus, walls that are in same 
line of sight as a sprite object and also have a smaller distance to the user from the 
sprite object are determined to obscure the sprite object. After the sprite object is 



wo 00/20987 PCT/IB99/01 799 

clipped (if necessary), any visible portions of the sprite image are mapped to the 
clipped screen dimension of the 3D viewport in operation 1506. After the first sprite 
object has been rendered and additional sprites are sequentially processed in the same 
manner. The process 1 1 10b for rendering sprite objects that are not located on wall 
then ends. 

Figure 16 is a flowchart illustrating the operation 5550 of Figure 7 for 
updating the local links viewer in accordance with one embodiment of the present 
invention. Initially, names of nearby links are obtained from the link attribute layers 
in operation 1602. Nearby links are defined as links that are substantially near the 
current location within the viewer 100. For example, the nearby links may be defined 
as any links that are currently being displayed within the viewer 100. Alternatively, 
the nearby links may be defined as links that are directly adjacent (e.g., on adjacent 
tiles) to the current location within the 3D representation. Alternatively, links that are 
within a predefined distance of the current location or tile may be displayed. 

After the nearby link names are obtained, the links' names are displayed 
within the local link viewer based on positions of the nearby links' positions relative 
to the user*s current location in operation 1604. In other words, the displayed links' 
positions within the local link viewer correspond to the nearby links' positions within 
the 3D representation. After the names are displayed, the process 5550 ends. 

Figure 17 is a flowchart illustrating the operation 5560 of Figure 7 for 
updating the 2D viewport in accordance with one embodiment of the present 
invention. Initially, it is determined whether the 2D map or 2D viewport image has 
changed since the last update in operation 1702. The 2D map may need to be updated 
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for any number of reasons. For example, the current location within the 3D image 
may have changed, and thus, the 2D map s location indicator 149 is drawn in a new 
position on the 2D map 1 06. 

If the map has changed, a new 2D image is drawn into the 2D viewport to 
5 reflect the changes since the last update in operation 1 704. After the new 2D image is 
drawn or if it is determined that the map has not changed since the last update, the 
position of the current location indicator's 149 position is calculated within the 2D 
viewport based on a new position within the 3D representation in operation 1706. 

Likewise, the current location indicator's direction within the 3D viewport is 
also calculated based on the user's direction within the 3D representation in operation 
1708. As discussed above, the user's direction is defined as the direction within the 
3D representation that the user is looking. For example, if the user is looking towards 
the southern portion of the 3D representation, the user's directions is "south". A 
location indicator image is then selected that is appropriate for the calculated direction 
15 in 1710. The location indicator is then drawn within the 2D viewpon in operation 
1712 based on the calculated user position and the selected location indicator image. 
The process 5560 for rendering the 2D viewport then ends. 

Figure 1 8 is a flowchart illustrating a process 6200 for updating the world list 
in accordance with one embodiment of the present invention. As described above, the 
20 world list is a hierarchical list of 3D representations that are each associated with a 
parent 3D representation or a marker. An update of the world list may be performed 
in response to any suitable user input or automated request. For example, the user 



BNSDOCID: <WO_00a0987A2J_> 



wo 00/20987 PCT/IB99/01 799 

may select the "world" button 127 and then select a 3D representation from the 
displayed world list. 

Initially, local world lisi data is loaded in operation 1802. The loaded list is 
then parsed for sublist place holders in operation 1604. A place holder is configured 
5 to reference another list that may not be located on the local drive (e.g. located at a 
particular URL site). For each sublist place holder, the referenced sublist is then 
loaded and appended to the local world list data in operation 1 808. 

After the referenced external sublist(s) are appended to the local list, the 
currently displayed 3D representation's name and position within the world list are 
10 determined in operation 1810. The list is then displayed with the current 3D 
representation's name visible and/or highlighted in operation 1812. This feature 
allows the user to quickly assess which 3D representation is currently being displayed 
and how such 3D representation fits within the list. 

If another 3D representation's name is selected from the list, the list is closed 
15 and the new 3D representation is loaded in operation 1814. In other words, the 
currently displayed 3D representation is shut down and a new 3D representation is 
loaded that is associated with the selected name. The updating process 6200 then 
ends. 

Figure 19 is a flowchart illustrating the process 1900 for loading a selected 
20 link in accordance with one embodiment in the present invention. A particular link 
may be selected in a number of ways. For example, a link may be directly typed 
within the entry box 1 1 1 (see Figure 1): selected by clicking on a link within the 2D 
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viewport 106: selecting by clicking on a link within the local links viewer 113; or 
loaded by walking into a link within the 3D representation {e.g., walking into a 3D 
building that is associated with a URL site). 

When a link is selected or activated, a link name is parsed to determine the 
link type in operation 1902. It is then determined whether the link should be handled 
by a browser in operation 1904. If a browser should be used, the browser is loaded 
into the 3D viewport 103 in operation 1906. The link is then passed to the browser 
for display in operation 1908. 

If a browser is not going to be used, it is then determined whether the link is 
associated with a 3D representation in operation 1910. If the link references a 3D 
representation, the associated 3D representation is loaded in operation 1914. 
However, if the link references another type of object, the link is passed to another 
mechanism that is capable of handling the particular linked object in operation 1912. 
For example, if the linked object is an Microsoft Word document, Microsoft Word 
may be executed on the document. The process 1900 for loading a link then ends. 

Figure 20 is a flowchart illustrating the operation 6540 of Figure 19 for 
loading the 3D representation in accordance with one embodiment of the present 
invention. Initially, the currently displayed 3D representation is shut down in 
operation 2002. A data structure associated with the new 3D representation is then 
loaded in operation 2004. Any ancillary files associated with the data structure are 
then determined and loaded in operation 2006. For example, a data structure may 
include image files (e.g., floor, wall, and ceiling images) for rendering various objects 
within the 3D representation. A new 3D representation associated with the loaded 
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data structure is then initialized in operation 2008, The process then proceeds to 
operation 5500 for rendering objects within the viewer 100, which process is 
described in Figure 7. 

The viewer 100 may include additional mechanisms for handling 3D 
representations in conjunction with an email application. For example, the viewer 
100 may allow the user to send and receive emails that include a 3D representation, 
along with all of its associated files. Figure 21 is a flowchart illustrating the process 
2100 for sending and receiving 3D email in accordance with one embodiment of the 
present invention. 

Initially, a 3D representation is identified in operation 2102. This may be 
accomplished by any suitable user interface mechanism. For example, the user may 
identify the 3D representation by simply opening a particular 3D representation from 
the "file" pull down menu within the viewer 100. After the 3D representation is 
identified, 3D email is activated in operation 2104. e.<i., by pushing an "email" button 
(not shown). 

A "send" or "receive" operation is then identified in operation 2106, The 
identification may implement any suitable mechanism. e,g., determining whether a 
"send" or "receive" email button has been selected. If a "send" operation is identified, 
the identified 3D representation is packaged as an email object, along with all of its 
associated files. That is. the package will contain all necessary files and/or links for 
displaying the 3D representation. An email address is then obtained in operation 
2110, and the email object is then sent to the email address in operation 2112. 
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If a "receive" operation is identified, a received 3D package is obtained in 
operation 2118. The viewer 100 may be configured to access the user's email post 
office from another email application, such as Microsoft Outlook. Altematively, the 
viewer 100 may itself include email capabilities for receiving and sending email. 

5 After the 3D package is obtained, it is unpacked in operation 2114. In other 

words, the data structure and associated image files may have to be separated fi-om the 
received email object. A 3D representation of the received data structure and files is 
then displayed within the viewer 100 as a 3D email in operation 2116. After the 3D 
email is displayed (or the 3D email is sent), the 3D email process 2100 ends. 

10 Web Display Embodiments 

The above described mechanisms may be utilized to display one or more web 
pages within a web display window, in addition to the above described 3D world 
within the 3D viewport 103. Figure 22 is a screen shot of the viewer 2201 in 
accordance with one embodiment of the present invention. As shown, the viewer 

15 2201 includes a 3D viewport 2200 presenting a 3D world, a web display 2204 
presenting a web page, a 2D viewport 2212 presenting a birdseye view of the 3D 
world, an emblem 2208. a local link display 2210, and a caption bar 2214 presenting 
a token. Several embodiments of the 3D viewport. 2D viewpon, emblem, local link 
display are described above, while several embodiments of the token are described 

20 below with reference to Figures 28-3 1 . 

Web pages may be associated with specific locations or tiles within the 3D 
world or the entire 3D world itself In one embodiment, as a user enters or clicks a 
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location within the 3D world, a corresponding web page is separately displayed 
within the web window. Thus, different web pages may be displayed while 
navigating within the 3D world. Alternatively, as the user moves within the 3D 
world, a web page may be constantly displayed within the web display. 

5 Although the web display window is described in terms of web pages, the web 

display may present any suitable information that enhances the 3D world experience. 
For example, the web display may include anything that may be generated by HTML 
commands. The web display may include a secondary 3D world or a secondary 2D 
representation associated with the main 3D world. The web display may simply list 
10 information (e.g,, product lists) pertaining to what is currently being shown in the 3D 
world. The web display may be in the form of streaming audio and/or video that is 
related to the 3D world. 

The web display may also include control mechanisms, such as buttons, for 
controlling various aspects of the 3D world, or other windows within the viewer. The 
15 control mechanisms may also include any suitable user interface devices. For 
example, the control mechanisms may include a button, a slider, a dial, pull down 
menu, a entry window, a switch, or any combination thereof 

Additionally, the web page may also be controlled. For example, the 3D 
world may control various content of the web page. In other embodiments, the web 
20 display may include customer help sessions, (e,g.. via live video, live audio, or email) 
to give the user a direct link to a customer support person. 

The information displayed within the web display may be available through 
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the Internet, or any other suitable network site. Alternatively, the information may be 
obtained from a local disk or CDROM on the user s computer. The relative size of 
the various windows {e.g., the 3D view and the web display) may also be alterable 
(e.g., the web display may be larger than the 3D view). Also, the size of each window 
5 may be controlled by the Borg data structure. The user may configure the size of each 
window, and the selected configurations are input into the data structure. 

Examples of 3D worlds and corresponding web displays are illustrated in 
Figures 22-27 in accordance with one embodiment of the present invention. In these 
examples, the user navigates through various areas of a 3D world that is in the form 
10 of a shopping mall. As the user approaches a particular object or area within the mall, 
information corresponding to the object or area is displayed within the web display. 

As shown in Figure 22. a 3D view of the mall interior 2200 is displayed 
within the 3D window. As shown, the mall includes an online music shop billboard 
2206. When the user approaches the music shop billboard 2206 (or clicks on the 

15 billboard), an online shopping web page 2204 corresponding to the particular music 
shop 2206 is displayed within the web display. In other words, the web display 
connects the user to the music shop's online shopping web page. The user may then 
select songs to be played (e.g., via RealPlayer®) or to be bought over the Internet. 
The web page may also utilize streaming audio that is played while navigating 

20 through the 3D world. 

In this embodiment, the 3D mall contains shops that have rooms dedicated to 

particular themes. For example, a music company may have rooms for different 

artists, and the rooms are decorated with various paraphernalia related to the artist. 
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The room may be decorated with posters of the artist and records by the artist. As the 
user approaches each object, relevant information about such object may be presented 
in the web display. For example, as the user approaches or selects an album, 
streaming audio may be played or a list of songs may be presented within the web 
display. As illustrated in Figure 23, the room contains a 3D representation of the 
artist 2302, such as Lenny Kravitz, for whom the room is constructed. The web 
display includes a list of Lenny's songs 2304 with hyperlinks to more information 
about particular songs or to play the song as the 3D world is navigated. 

As shown in Figure 24. the 3D world includes a 3D image of a ''chat booth" 
2402 that looks like a telephone booth. As the user approaches or selects the chat 
booth 2402 within the 3D view, the web display connects to a live chat page 2404. 
As shown in Figure 25, various portions or areas of the 3D world 2502 may be 
navigated and controlled via the web display 2504. That is, the web display includes 
buttons for controlling how the 3D world is displayed. For example, a tour button is 
displayed so that the user may select a tour within the 3D world and several speed 
buttons are displayed so that the user may select a speed for navigating through the 
3D world). 

Thus, each web page may send web commands to the viewer 1 00 that affect 
how portions of the viewer are displayed. For example, the contents of the 3D world 
may change in response to the displayed web page. The web page may send 
commands to the viewer 1 02 in any suitable manner. In one embodiment, a web page 
may send a command in the form HTML links that are prefixed with borg://." This 
prefix is registered with the operating system. When a ''borg://" HTML link is 

39 



wo 00/20987 PCT/IB99/0 1 799 

received, the viewer 102 interprets the link. In one embodiment, all the buttons on 
the viewer 100 may also be activated and/or modified by a web page. Additionally, 
the web page may be controlled tlirough COM (component object model) objects. 
The COM objects include a plurality of methods that may be accessed. 

5 Figure 26 shows a shop 2602 within the 3D viewer. The shop 2602 contains 

3D items that are for sell. As the user approaches various objects for sale, the web 
display shows a standard online shopping page 2604 for purchasing such items (e.g., 
adding the items to your shopping cart and checking out). In one embodiment, if the 
user clicks on an item within the 3D world (e.g.. a handbag), purchase information 

10 and order taking buttons are made available in the web display. As illustrated in 
Figure 27, the user may also enter an interior of a movie theatre 2702 within the 3D 
viewer. The movie theatre contains movie advertisements. As the user approaches a 
particular movie advertisement, the web display shows information 2704 about when 
the particular movie is playing. The web display may also utilize streaming video 

1 5 and/or audio for playing a portion of the particular movie. 

Web links to web pages may be associated with a 3D world in any suitable 
manner. 
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Token Embodiments 

Mechanisms may also be provided for designating who built a particular 3D 
world and/or ensuring that the originator of the 3D world is not distributing 3D 
worlds without authorization, e.g., from the licensor. In one embodiment, a token is 
created that will be inserted into the builder. The inserted token results in a particular 
phrase being displayed within the viewer. For exfimple, the inserted token may result 
in the phrase "for personal use only" or "unregistered builder" (as shown in the 
caption bar 2506 of Figure 25) being displayed within the viewer when a 3D world is 
generated by the licensee's builder. Otherwise, if the licensee is authorized to 
distribute 3D worlds, the licensee may select their own token plirsise, such as 
"CYBERWORLD Viewer - \\ wvv.cvherworldcorp.com " (as shown in the caption bar 
2214 of Figure 22).'' At least the token phrase is encrypted to ensure that it is 
unlikely to be easily copied by others or altered by the licensee after it is created. 

One function of these token mechanisms is lo provide ways to distinguish 
between unauthorized and authorized 3D worlds. Any 3D worlds that are publicly 
available {e.g., on the web) will be appropriately marked to indicate authorization 
status. The source of the unauthorized 3D world may also be readily traced. For 
example, the owner of the web site may be tracked or the token may identify the user 
{e.g.. via a serial number). Of course, any suitable marking may be utilized to 
determine which 3D worlds or buyers are authorized to generate publicly available 
3D worlds. 

Figure 28 is a flowchart illustrating a process 3600 for creating a token file in 
accordance with one embodiment of the present invention. Initially, a user phrase and 
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serial number is input in operation 3602. The serial number is optional. The phrase 
and serial number are then encr>'pted in operation 3604. Any suitable encryption 
algorithms may be utilized. A token file having the encrypted phrase and serial 
number is then output in operation 3606. The process 3600 then ends. 

The user phrase may be selected by the user of the builder or predefined based 
on the licensing rights of the user. For example, a user that is only entitled to use the 
builder for personal use may have the user phrase (e.g,, for personal use only) 
designated by the licensor. Other users may be allowed to select a personalized 
phrase that is distinguishable from other user's phrases (e.g., a tradename). Any 
number of tokens may be generated for a particular user or builder by implementing 
process 3600. Each user phrase may be included within the same or different token 
file. 

After the token file is created. Figure 29 illustrates how the token is then 
installed within the associated builder in accordance with one embodiment of the 
present invention. Initially, the token file is opened in operation 3702. The contents 
are then read and validated in operation 3704. For example, a checksum may be 
utilized to check whether the token phrase has been altered by the user. If a serial 
number is used, it may then be confirmed in operation 3706. In other words, the 
serial number is matched to the builder's serial number. This operation ensures that 
the token was not obtained by an unauthorized user with a nonmatching builder. 

The encr\'pted phrase and human readable phrase is then added to a token list 
in operations 3706 and 3708, respectively. The human readable phrase may be the 
user phrase itself, a shortened version of the user phrase, or any suitable string that 
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identifies the user phrase. Of course, when the user is only authorized to use the 
builder for personal use, only one plirase (and no human readable plirase) may be 
inserted into the token list indicating that the builder is for personal use only. The 
process 3700 for inserting the token within the builder then ends. 

Figure 30 is a flowchart illustrating the process 3800 for building a 3D world 
with the token in accordance with one embodiment of the present invention. Initially, 
a token is selected from the human readable phrases within the token file. The 
number and type of tokens that are available depends on the type of builder utilized 
and/or the particular rights of the associated user. For example, a builder that is 
authorized to distribute 3D worlds may include tokens that were previously chosen by 
the user. On the other hand, a builder that is for personal use only may include a 
single token that indicates this restriction through the viewer. The token may be 
selected by the user. Alternatively, the token may be automatically selected (e.g., if 
there is a single token). 

After the token is selected, the corresponding encrypted phrase is obtained in 
operation 3804. The encrypted phrase is then inserted within the Borg or world data 
structure that is used to generate and display the 3D world and web display. The 
process 3800 for building with the token then ends. 

Figure 39 is a flowchart illustrating a process 3900 of rendering the token 
within the viewer in accordance with one embodiment of the present invention. 
Initially, the Borg structure is loaded into the viewer in operation 3902. The user 
phrase is then decrypted in operation 3904. It is then determined whether the 
decrypted phrase is valid in operation 3906. If the phrase is valid, the phrase is 
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displayed, for example, in the caption bar of the viewer in operation 3910 and the 3D 
world is displayed in operation 3914. If the phrase is not valid, an invalid token" 
message may be displayed in operation 3912 and the process 3900 ends. 
Alternatively, the 3D world may also be displayed in operation 3914, along with the 
" invalid token" message, before ending the process 3900. 

Any suitable parameters may be checked to determine validity. For example, 
a checksum may be determined again to check whether the token phrase has been 
altered. Various other checks may be performed. In one embodiment, it may be 
determined whether the token has been used for a longer time than a specified time 
limit. The builder's version may also be checked to determine v/hether the builder is 
out of date and no longer valid. The process 3900 of rendering the token then ends. 

The above described token mechanisms may be applied to any suitable audio 
visual display generator where it is desirable to identify a user of the such generator. 
For example, any software that generates an audiovisual display may include the 
above token mechanisms to indicate whether the user is authorized to distribute the 
generated audiovisual displays. 

Advertisement Banner Embodiments 

In other embodiments, the viewer also includes an advertising banner. The 
advertising banner may be controlled either by the user or by a third party, such as the 
licensor of the builder. For example, the licensor generates an advertisement banner 
that is automatically displayed within a viewer that is utilized by a low-end builder. 
In this embodiment, the builder may include mechanisms for automatically linking to 
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a specific advertisement web page that is controlled by the third party or licensor. 
Alternatively, the user may be given control of the advertisement web page, for 
exaniple, in high-end builders. In other words, the user may be allowed to generate 
personal advertisement banners. 

The advertisement bamier may have its own separate window or may replace 
one of the windows of the viewer. For example, the advertisement banner may be 
substituted in place of the local links display window 1 1 3 of figure 1 . 

Additionally, other aspects of the viewer may be controlled either by the user 
or by a third party (e.g., the licensor of the builder). For example, the emblem 109 of 
Figure 1 may be controlled by the licensor of the builder. The licensor may wish to 
advenise their logo in the emblem window 109. 

Although the foregoing invention has been described in some detail for 
purposes of clarity of understanding, it will be apparent that certain changes and 
modifications may be practiced within the scope of the appended claims. It should be 
noted that there are many alternative ways of implementing both the process and 
apparatus of the present invention. Accordingly, the present embodiments are to be 
considered as illustrative and not restrictive, and the invention is not to be limited to 
the details given herein, but may be modified within the scope and equivalents of the 
appended claims. 
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CLAIMS 

WHAT IS CLAIMED IS: 

1. A viewer user interface adapted to present information that is organized 
into a discrete chunk of space to a user, wherein the information potentially includes 

5 links and/or objects that are associated with specific locations in the space, the viewer 
user interface comprising: 

a first viewport for presenting a three dimensional representation of the space 
from the viewpoint of a position within the space; 

a second viewport for presenting a two dimensional birdseye view representation 
10 of the discrete chunk of space; and 

a third viewport for presenting an identifier that identifies the discrete chuck of 
space is associated with. 

2. A viewer user interface adapted to present information that is organized 
15 into a discrete chunk of space having a plurality of sections to a user, wherein the 

information potentially includes links and objects including at least some of images, 
sound clips, video clips and sprites that are associated with specific locations in the 
space, the viewer user interface comprising: 

a first viewport for presenting a three dimensional representation of the space 
20 from the viewpoint of a moveable reference position within the space: 
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a second viewport for presenting a two dimensional birdseye view representation 
of the discrete chunk of space, the second viewport being arranged to indicate the current 
reference position; 

a third viewport for presenting an identifier that identifies the discrete chuck of 
space is associated with: 

an entry bar arranged to identify the discrete chunk of space that the first and 
second viev^orts are presenting; 

a local link viewer arranged to identify links in selected locations that are adjacent 
to the current reference position: and 

a status box arranged to present status information. 

3. A viewer user interface adapted to present information that is organized 
into a discrete chunk of space having an array of tiles to a user, wherein the information 
potentially includes links and assets including at least some of sound clips, video clips 
and sprites that are associated with specific locations in the space, the viewer user 
interface comprising: 

a first viewport for presenting a three dimensional representation of the space 
from the viewpoint of a moveable reference position within the space: 

a second viewport for presenting a two dimensional birdseye view representation 
of the discrete chunk of space, the second viewport being arranged to indicate the current 
reference position; and 

a local link viewer arranged to identify links in selected tiles that are adjacent to a 
current reference tile that includes the current reference position. 
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4. A method of display ing a three dimensional representation, comprising: 
changing a first three dimensional view of the three dimensional representation to 

a second three dimensional view of the three dimensional representation after receiving 
an input signal indicating movement within the three dimensional representation; and 
5 changing a first position of a reference object within a discretely sized two 

dimensional representation to a second position, the reference object being associated 
with the three dimensional representation, the first position corresponding to the first 
three dimensional view and the second position corresponding to the second three 
dimensional view. 

10 

5. A method as recited in claim 4, further comprising displaying names of 
links that are proximate to the second three dimensional view within a local link viewer. 

6. A method as recited in claim 4, further comprising: 

15 displaying a hierarchical list of related three dimensional representations after 

receiving an input signal indicating selection of the hierarchical list, wherein the 
currently displayed three dimensional representation is displayed within the hierarchical 
list; and 

displaying a list of links associated with the currently displayed three dimensional 
20 representation after receiving an input signal indicating selection of the associated links. 

7. A method as recited in claim 6, further comprising activating a selected 
link after receiving an input signal indicating selection of the link, wherein the link is 
selected from the displayed hierarchical list. 
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8. A method as recited in any of claims 4-7, further comprising activating a 
selected link after receiving an input signal indicating selection of the link 

5 9. A method as recited in claim 8, wherein the link is selected from the three 

dimensional representation. 

10. A method as recited in claim 8, wherein the link is selected from the two 
dimensional representation. 

10 

11. A method as recited in any of claims 4- 1 0. further comprising changing a 
first orientation of the reference object to a second orientation within the two 
dimensionah wherein the first orientation corresponds to a first viewing directions of the 
first three dimensional view, the second orientation corresponds to a second viewing 

15 direction of the second three dimensional view, the first position of the reference object 
corresponds to a first current position of the first three dimensional view and second 
position of the reference object corresponds to a second current position of the second 
three dimensional view, 

20 12. A method as recited in any of claims 4-1 K further comprising changing a 

display characteristic of the three dimensional representation after receiving an input 
indicating modification of a display characteristic. 

13. A display system for displaying a three dimensional world, comprising: 
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a three dimensional display window arranged to output at least a portion of the 

three dimensional world; and 

a two dimensional display window arranged to output a two dimensional 

representation of the three dimensional world that includes a position indicator associated 

with the portion of the three dimensional world that is displayed in the three dimensional 

display window. 



14. A computer readable medium containing program instructions for 
displaying a three dimensional representation comprising: 

computer readable code for changing a first three dimensional view of the three 
dimensional representation to a second three dimensional view of the three dimensional 
representation after receiving an input signal indicating movement within the three 
dimensional representation; 

computer readable code for changing a first position of a reference object within a 
discretely sized two dimensional representation to a second position, the reference object 
being associated with the three dimensional representation, the first position 
corresponding to the first three dimensional view and the second position corresponding 
to the second three dimensional view; and 

a computer readable medium that stores the computer codes. 

15. A viewer user interface adapted to present information that is organized 
into a discrete chunk of space to a user, wherein the information potentially includes 
links and/or objects that are associated with specific locations in the space, the viewer 
user interface comprising: 
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a first viewport for presenting a three dimensional representation of the space 
fi-om the viewpoint of a position within the space: and 

a second viewport for presenting information related to at least a portion of the 
three dimensional representation. 

5 

16. A viewer user interface as recited in claim 15 wherein the information 
within the second viev^ort is also related to the position within the space. 

17. A viewer user interface as recited in claim 15 or 16 wherein the 
10 information originates from a URL site on the world wide web. 

18. A viewer user interface as recited in claim 15 wherein the information 
within the second viewport is a secondary 3D representation. 

15 19. A viewer user interface as recited in claim 15 wherein the information 

within the second viewport is additional information that is not displayed within the first 
viewport. 

20. A viewer user interface as recited in claim 15 wherein the information 
20 within the second viewport is selected from a group consisting of streaming video, 
streaming audio, live video, and live audio. 
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21. A viewer user interface as recited in claim 15 wherein the information 
within the second viewport contains at least one control mechanism for controlling 
aspects of the 3D representation. 

5 22. A viewer user interface as recited in claim 21 wherein at least one of the 

control mechanisms of the second viewport are arranged to control how the 3D 
representation is displayed. 

23. A viewer user interface as recited in claim 21 wherein at least a one of the 
10 control mechanisms is a user controllable interface. 

24. A viewer user interface as recited in claim 23 wherein the user 
controllable interface is selected from a group consisting of a button, a slider, a dial, pull 
down menu, a entry window, and a switch. 

15 

25. A viewer user interface as recited in any of claims 15-24 wherein a size of 
the first viewport and a size of the second viewport are both configurable. 

26. A viewer user interface adapted to present information that is organized 
20 into a discrete chunk of space having a plurality of sections to a user, wherein the 

information potentially includes links and objects including at least some of images, 
sound clips, video clips and sprites that are associated with specific locations in the 
space, the viewer user interface comprising: 
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a first viewport for presenting a three dimensional representation of the space 

from the viewpoint of a moveable reference position within the space; and 

a second viewport for presenting information related to the current reference 

position. 

5 

in. A viewer as recited in claim 26 further comprising a third viewport for 
presenting a two dimensional birdseye view representation of the discrete chunk of space, 
the second viewport being arranged to indicate the current reference position. 

10 28. A viewer user interface as recited in claim 27 wherein the information 

presented within the second viewport is a secondary two dimensional representation. 

29. A viewer as recited in any of claims 26-28 further comprising a fourth 
viewport for presenting a identifier that the discrete chuck of space is associated with. 



15 



20 



30. A viewer as recited in claim 29 wherein the identifier is user configurable. 



31. A viewer as recited in claim 29 wherein the identifier is nonalterable by a 

user. 

32. A viewer as recited in any of claims 26-3 1 fiirther comprising an entry bar 
arranged to identify the discrete chunk of space that the first and second viewports are 
presenting. 
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33. A viewer as recited in any of claims 26-32 further comprising a local link 
viewer arranged to identify links in selected locations that are adjacent to the current 
reference position; and 

5 34. A viewer as recited in any of claims 26-33 further comprising a status box 

arranged to present status information. 

35. A viewer as recited in any of claims 26-34 further comprising an 
advertising window to present advertisement information. 

10 

36. A viewer as recited in claim 35 wherein the advertisement information 
originates from a URL site. 

37. A viewer as recited in claim 35 wherein the URL site is selectable by a 

15 user. 

38. A viewer as recited in claim 35 wherein the URL site is fixed such that a 
user is prohibited from replacing the URL site with another URL site. 

20 39. A method of displaying a three dimensional representation and related 

information, comprising: 

displaying a first portion of the three dimensional representation within a fu^t 
viewport; 
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displaying a first set of information related to the first portion of the three 

dimensional representation; 

displaying a second portion of the three dimensional representation within the 

first viewport in response to receiving an input signal indicating movement within the 
5 three dimensional representation; and 

displaying a second set of information related to the second portion of the three 

dimension representation. 

40. A method as recited in claim 39, further comprising displaying names of 
10 links that are proximate to the second three dimensional view within a third viewport. 

41 . A method as recited in claims 39 or 40, further comprising: 

displaying a hierarchical list of related three dimensional representations after 
receiving an input signal indicating selection of the hierarchical list, wherein the 
15 currently displayed three dimensional representation is displayed within the hierarchical 
list; and 

displaying a list of links associated with the currently displayed three dimensional 
representation after receiving an input signal indicating selection of the associated links. 

20 42. A method as recited in claim 41, fiirther comprising activating a selected 

link after receiving an input signal indicating selection of the link, wherein the link is 
selected fi-om the displayed hierarchical list. 
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43. A method as recited in claim 41, further comprising activating a selected 
link after receiving an input signal indicating selection of the link 



44. A method as recited in any of claims 39-43 further comprising changing a 
5 display characteristic of the currently displayed portion of the three dimensional 

representation after receiving an input from a source of the information indicating 
modification of the display characteristic. 

45. A method as recited in claim 44 wherein the source is a URL address. 

10 

46. A display system for displaying a three dimensional world, comprising: 

a three dimensional display window arranged to output at least a portion of the 
three dimensional world; and 

a web display window arranged to output a web page corresponding to at least the 
15 portion of the three dimensional world output in the three dimensional display window. 

47. A display system as recited in claim 46 further comprising a two 
dimensional display window arranged to output a two dimensional representation of the 
three dimensional world that includes a position indicator associated with the portion of 

20 the three dimensional world that is displayed in the three dimensional display window. 

48. A computer readable medium containing program instructions for 
displaying a three dimensional representation and related information, comprising: 
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computer readable code for displaying a first portion of the three dimensional 
representation within a first viewport; 

computer readable code for displaying a first set of information related to the first 
portion of the three dimensional representation; 

computer readable code for displaying a second portion of the three dimensional 
representation within the first viewport in response to receiving an input signal indicating 
movement within the three dimensional representation; and 

computer readable code for displaying a second set of information related to the 
second portion of the three dimension representation; and 

a computer readable medium that stores the computer codes. 

49. A method of displaying a security token within a displayed output of an 
audiovisual display generator, the method comprising: 

encrypting a user phrase, the user phrase identifying a user of the audiovisual 
display generator; 

adding encrypted phrase to a token list; 
selecting the encrypted phrase from the token list; 
decrypting the user phrase from the encrypted phrase; and 

displaying the user phrase within the displayed output of the audiovisual 
generator such that the user of the audiovisual display generator is identified. 

50. A method as recited in claim 49 wherein the encrypted user phrase is 
automatically selected by the audiovisual generator such that the user is prohibited from 
altering the user phrase. 
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51. A method as recited in claim 49 wherein the encrypted user phrase is 
selectable by the user. 

5 52. A method as recited in claim 51 wherein the encrypted user phrase is 

associated with a human readable phrase so as to facilitate user selection of the encrypted 
user phrase. 

53. A method as recited in any of claims 49-52 wherein the user phrase is 
10 displayed within a caption b£ir of the displayed output. 

54. A method as recited in any of claims 49-53 further comprising 
determining whether the decrypted user phrase is valid and wherein the user phrase is 
only displayed when it is valid. 

15 

55. A method as recited in claim 54 wherein the validity determination is 
based on whether or not the decrypted user phrase matches the encrypted user phrase or 
has been altered since being encrypted. 

20 56. A method as recited in claim 54 further comprising matching the user 

phrase with a serial number of the audiovisual display generator and wherein the validity 
determination is based on whether or not the serial number of the audiovisual display 
generator matches the decrypted user phrase. 
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57. A computer readable medium containing program instructions for 
displaying a security token within a displayed output of an audiovisual display generator, 
the computer readable medium comprising: 

computer code for encrypting a user phrase, the user phrase identifying a user of 
5 the audiovisual display generator; 

computer code for adding encrypted phrase to a token list; 
computer code for selecting the encrypted phrase from the token list; 
computer code for decrypting the user phrase from the encrypted phrase; 
computer code for displaying the user phrase within the displayed output of the 
10 audiovisual generator such that the user of the audiovisual display generator is identified; 
and 

a computer readable medium that stores the computer codes. 
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